Skip to content

Unify command recognition for PhysicalDevice and DebugPort - #73

Merged
dccote merged 19 commits into
masterfrom
table-driven-debug-port
May 18, 2026
Merged

dccote merged 19 commits into
masterfrom
table-driven-debug-port

Conversation

@dccote

@dccote dccote commented Mar 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR delivers the full dual-role commands dict pattern — both the client send-side and the mock recognition-side are now driven by the same Command objects on each device.

Phase 1: Command recognition unification (commits e6e88ad…91e7a8f)

  • Fix DebugPort bugs (endpoint indexing, reentrant locking) and add multi-endpoint support
  • Add TableDrivenDebugPort for declarative mock device creation driven by Command objects
  • Add matches(), extractParams(), formatResponse() to Command/TextCommand/DataCommand so the same objects define both client send protocol and debug port recognition
  • Remove BinaryCommandEntry/TextCommandEntry dataclasses — replaced by a single polymorphic loop over Command objects
  • Support named parameters: (?P<name>...) regex groups for TextCommand, requestFields/responseFields for DataCommand — params become dicts instead of positional tuples
  • Add class-level commands dicts to SutterDevice and IntellidriveDevice using named parameters throughout
  • Add debug-mode tests for IntellidriveDevice (9 tests exercising init, home, move, orientation, status flags)

Phase 2: Send-side migration (commits 812bcee, 42bff00)

Previously on a separate send-side-migration branch; rebased on top of Phase 1.

  • Add buildSendData(**params) and unpackReply(replyBytes) to DataCommand plus sendFormat/sendFields/sendDefaults constructor params, so the commands dict can now also build outgoing packets (not just recognize incoming ones)
  • Rewrite SutterDevice.doGetPosition, doMoveTo, doHome, and work to use the new sendCommand(name, **params) helper — no more hand-built struct.pack() calls
  • Add WORK command to the Sutter commands dict and to DebugSerialPort
  • 5 new TestDataCommandSendSide tests in testCommandRecognition.py
  • Rename TextCommand.text → text_format (it's a format string, not plain text) across commands.py, debugport.py, devicemanager.py, echodevice.py, intellidrivedevice.py, integradevice.py, and all relevant tests. DeviceCommand.text in diagnostics.py is a separate NamedTuple and was NOT renamed.

What this unlocks

After this PR lands, the commands dict on a PhysicalDevice is the single source of truth for the protocol — adding a new device becomes "declare a commands = {...} dict" instead of writing hand-packed send logic AND a separate mock. The CLAUDE.md guidance that currently flags this pattern as "experimental, do not adopt for new devices" can be updated to "use the commands dict for new devices."

Test plan

  • 36 unit tests in testCommandRecognition.py pass (31 from Phase 1 + 5 send-side tests)
  • 12 testTableDrivenDebugPort.py tests pass
  • 7 testSutterDevice.py tests pass (now exercising the send-side migration)
  • 9 Intellidrive debug device tests pass
  • Full relevant suite: 139 passed, 29 skipped (skipped are hardware-gated)

History note

Phase 2 was sitting on a local-only send-side-migration branch dated 2026-03-01 that never got pushed. Discovered during a session cleanup today, rebased cleanly onto current table-driven-debug-port tip, fast-forward pushed (no force-push needed).

🤖 Generated with Claude Code

dccote and others added 3 commits March 1, 2026 16:15
Fix flush() writing to nonexistent attribute, fix endPointIndex unbound
when endPoint is not None, add numberOfEndPoints parameter for USB
devices, add defaultTimeout attribute, remove broken DebugDataCommand.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace manual if/elif byte-parsing in debug ports with a table-driven
approach: declare command entries (binary or text) and implement only
device logic in process_command(). Migrate Sutter's DebugSerialPort,
create Intellidrive's missing DebugSerialPort, and add 12 new tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ck sides

Add matches(), extractParams(), formatResponse() to Command/TextCommand/DataCommand
so the same objects define both the client protocol and debug port recognition.
Remove BinaryCommandEntry/TextCommandEntry dataclasses from debugport.py, replacing
them with a single polymorphic loop over Command objects in TableDrivenDebugPort.
Update SutterDevice and IntellidriveDevice to use class-level commands dicts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dccote dccote changed the title Add TableDrivenDebugPort for declarative mock devices Unify command recognition for PhysicalDevice and DebugPort Mar 1, 2026
dccote and others added 6 commits March 1, 2026 17:37
Add TestIntellidriveDebugDevice with 9 tests exercising init, home, move,
orientation, register setup, and status flags without hardware connected.
Set terminator to b'\r' on DebugSerialPort so readString stops correctly
on carriage-return-terminated responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TextCommand.extractParams() now returns a dict when the matchPattern uses
named groups (?P<name>...), and formatResponse() accepts dicts for named
template parameters. Anonymous groups and positional tuples still work.

Add class and method docstrings to TableDrivenDebugPort with examples
showing named parameters for self-documenting command definitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use (?P<register>...), (?P<value>...), (?P<mode>...) in matchPatterns
and {register}, {value}, {mode} in text/responseTemplate fields.
process_command now uses params["register"] instead of params[0].

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add requestFields and responseFields to DataCommand, mapping struct
fields to names so extractParams returns a dict and formatResponse
accepts a dict. Sutter process_command now uses params['x'], params['y'],
params['z'] and returns {'header': ..., 'x': ..., 'y': ..., 'z': ...,
'terminator': ...}.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document the module, all four classes (Command, TextCommand,
MultilineTextCommand, DataCommand), and their key methods. Covers
both the send side and recognition side roles, with parameter
descriptions and examples.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every PhysicalDevice defines a commands dict that serves both sides:
sending commands to real hardware and creating mock debug ports that
receive those same commands and reply in the correct format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dccote
dccote marked this pull request as draft May 18, 2026 15:33
dccote and others added 4 commits May 18, 2026 12:17
Add buildSendData()/unpackReply() to DataCommand so the same commands
dict that drives the mock debug port also builds outgoing packets.
Rewrite doGetPosition, doMoveTo, doHome, and work to use the new
sendCommand() helper, eliminating hand-built pack() calls. Add WORK
command to the commands dict and DebugSerialPort.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The parameter is a format string, not plain text. Rename it everywhere:
TextCommand/MultilineTextCommand __init__, all self.text references,
all call sites across device classes, debug ports, and tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The project's test files follow a 'testFooBar.py' naming convention
(no underscore separator). Pytest's default 'python_files' pattern
is 'test_*.py', so 'pytest hardwarelibrary/tests/' returned zero
collected tests -- tests could only be run by naming files
explicitly. That had become a real friction now that the suite
crosses 500 tests.

Added a [tool.pytest.ini_options] section to pyproject.toml:

- testpaths = ["hardwarelibrary/tests"]  -- so plain 'pytest' works
- python_files = ["test*.py"]            -- match the project's naming
- norecursedirs = ["deprecated"]         -- skip unmaintained legacy
                                            tests that error on import

After this, 'pytest' from the repo root collects 476 tests and
reports 263 passed, 213 skipped (hardware-gated), 0 failures.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dccote
dccote marked this pull request as ready for review May 18, 2026 16:32
dccote and others added 3 commits May 18, 2026 12:42
Restructures EchoDevice to follow the single-class-with-sentinel
pattern already used by SutterDevice and IntellidriveDevice rather
than maintaining a separate top-level DebugEchoDevice class.

EchoDevice changes:
- doInitializeDevice now branches on serialNumber == "debug" to
  pick between SerialPort (real FTDI) and DebugSerialPort (mock).
- Adds a nested DebugSerialPort(TableDrivenDebugPort) class that
  reuses the device's commands dict and implements process_command
  for ECHO1/ECHO2/ECHO3.
- DebugEchoDevice class deleted -- callers now use
  EchoDevice(serialNumber="debug") instead.

Test changes:
- TestDebugEchoPhysicalDevice now inherits BaseTestCases.TestPhysicalDeviceBase
  directly (matching TestSutterPhysicalDevice) rather than from
  TestEchoPhysicalDevice. The previous inheritance ran
  TestEchoPhysicalDevice.setUp via super().setUp(), which always
  raised SkipTest because no real FTDI hardware was attached.
  Debug tests therefore never actually ran. They run now, with
  11 passing and 2 USB-enumeration-dependent tests still skipping.
- testEchoCommands duplicated onto TestDebugEchoPhysicalDevice since
  it no longer inherits from TestEchoPhysicalDevice.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the ad-hoc CoboltDebugSerial class with a nested
DebugSerialPort(TableDrivenDebugPort) inside CoboltDevice, and
moves the protocol definition into a class-level 'commands' dict.

Eleven commands captured: GET_POWER, GET_REQUESTED_POWER, SET_POWER,
GET_ON_OFF, TURN_ON, TURN_OFF, GET_SERIAL_NUMBER, TURN_AUTOSTART_ON,
TURN_AUTOSTART_OFF, GET_AUTOSTART, GET_INTERLOCK. Each is declared
once with text_format/replyPattern (for the device send-side) plus
matchPattern/responseTemplate (for the debug port to recognize and
respond).

The doX methods now use commands["NAME"].send(port=self.port) and
read .matchGroups for the parsed values, eliminating the parallel
hand-rolled regex dispatch that lived in CoboltDebugSerial.write
(~120 lines of inline pattern matching).

The DebugSerialPort holds the mock device state (power, isOn,
autostart, serialNumber, interlock) and process_command returns
either a dict (for templated responses) or a raw "OK\r\n" / error
string. The background power-ramp thread (increasePowerSlowlyInBackground)
is preserved.

Manual sanity check: end-to-end roundtrip works via
CoboltDevice(bsdPath='debug') -- initializeDevice succeeds, all
queries return sensible values, doSetPower(0.5) converges through
the background ramp.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DebugEchoPort was a no-op subclass of DebugPort with an empty
__init__ that just called super(). It existed as a named alias for
"DebugPort used for echo testing", but added no behavior -- the
echo logic lives in DebugPort.processInputBuffers.

Removed:
- hardwarelibrary/communication/echoport.py (the alias)
- DebugEchoPort export from hardwarelibrary/communication/__init__.py
- TestDebugEchoPort class in testCommunicationPorts.py (was a
  duplicate of TestDebugPortDefaultsToECho, with the only
  difference being which alias it instantiated)

Updated:
- TestSlowDebugEchoPort now instantiates DebugPort() directly
  instead of DebugEchoPort(). Same behavior; one less indirection.

Tests still pass (246 passed, 202 skipped on the full suite).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dccote and others added 2 commits May 18, 2026 13:14
Adds README-8-DebugPort.md (~340 lines) explaining the debug-port
infrastructure that PR #73 introduces:

- TL;DR and when to pick DebugPort vs TableDrivenDebugPort.
- Class hierarchy diagram (Mermaid) showing CommunicationPort ->
  DebugPort -> TableDrivenDebugPort and the four device-specific
  DebugSerialPort subclasses that adopt the pattern.
- Reference table of Command attributes by role (send / recognize
  / respond) for both TextCommand and DataCommand.
- Sequence diagram (Mermaid) walking through SutterDevice.doMoveTo
  end-to-end -- buildSendData, matches, extractParams, process_command,
  formatResponse, unpackReply -- showing the same Command object
  consulted on both sides of the wire.
- A 7-step recipe for adopting the pattern in a new device, with
  text-protocol and binary-protocol examples and pointers to the
  reference implementations.
- Edge cases / gotchas section (unmatched input, terminator
  requirements, replyDataLength, auto-derived match patterns).
- "Where it's used" table listing the current adopters plus the
  remaining candidates (IntegraDevice, OISpectrometer).

Also adds a pointer from README-5-Communication-ports.md so the
existing README chain leads readers here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dccote
dccote merged commit 2f38e70 into master May 18, 2026
@dccote
dccote deleted the table-driven-debug-port branch May 18, 2026 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant